Skip to contentSkip to conversation

Change in openvpn[master]: forked-test-driver: Show test output always

Openvpn-devel In review flichtenheld (Code Review) 3 participants 11 messages Dec 11, 2023 – Apr 3, 2024 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): gerrit.1702310034000.I11e0091482d9acee89ca018374cb8d96d22f8514@...2715...

#1 lore raw

Attention is currently required from: plaisthos.

Hello plaisthos,

I'd like you to do a code review.
Please visit

    http://gerrit.openvpn.net/c/openvpn/+/479?usp=email

to review the following change.

Change subject: forked-test-driver: Show test output always
......................................................................

forked-test-driver: Show test output always

We want to see the progress, especially for slow tests like t_client.sh.

Change-Id I11e0091482d9acee89ca018374cb8d96d22f8514

---
M forked-test-driver

1 file changed, 1 insertion(+), 1 deletion(-)

git pull ssh://gerrit.openvpn.net:29418/openvpn refs/changes/79/479/1

forked-test-driver +1 -1
index be73b80..1baaa84 100755
@@ -109,7 +109,7 @@
 # to ameliorate tests themselves also writing to the log file. Our tests
 # don't, but others can (automake bug#35762).
 : >"$log_file"
-"$@" >>"$log_file" 2>&1
+"$@" 2>&1 | tee -a "$log_file"
 estatus=$?
 
 if test $enable_hard_errors = no && test $estatus -eq 99; then

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 1
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-Attention: plaisthos <[email protected]...>
Gerrit-MessageType: newchange

#2 lore raw
In reply to a message not in the archive

Attention is currently required from: plaisthos.

flichtenheld has posted comments on this change. ( http://gerrit.openvpn.net/c/openvpn/+/479?usp=email )

Change subject: forked-test-driver: Show test output always
......................................................................

Patch Set 1:

(1 comment)

File forked-test-driver:

http://gerrit.openvpn.net/c/openvpn/+/479/comment/f56067ca_738d3288 :
PS1, Line 112: "$@" 2>&1 | tee -a "$log_file"
breaks exit status without set -o pipefail. Which is probably not portable... Setting to WIP until I have figured out a solution for that.

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 1
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-Attention: plaisthos <[email protected]...>
Gerrit-Comment-Date: Tue, 12 Dec 2023 13:16:32 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Gerrit-MessageType: comment

#3 lore raw
In reply to a message not in the archive

Attention is currently required from: plaisthos.

flichtenheld has posted comments on this change. ( http://gerrit.openvpn.net/c/openvpn/+/479?usp=email )

Change subject: forked-test-driver: Show test output always
......................................................................

Patch Set 3:

(1 comment)

This change is ready for review.

File forked-test-driver:

http://gerrit.openvpn.net/c/openvpn/+/479/comment/bf08f7d3_1455cd09 :
PS1, Line 112: "$@" 2>&1 | tee -a "$log_file"

Quoted 1 line

breaks exit status without set -o pipefail. Which is probably not portable... […]

Done

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 3
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-Attention: plaisthos <[email protected]...>
Gerrit-Comment-Date: Wed, 13 Dec 2023 10:49:18 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: flichtenheld <[email protected]...>
Gerrit-MessageType: comment

#4 lore raw
In reply to a message not in the archive

Attention is currently required from: flichtenheld.

plaisthos has posted comments on this change. ( http://gerrit.openvpn.net/c/openvpn/+/479?usp=email )

Change subject: forked-test-driver: Show test output always
......................................................................

Patch Set 5: Code-Review-1

(1 comment)

Patchset:

PS5:
The patch does not do what it promises to do. I cherry-picked this patch and the other forked-test-driver patch on my branch and in github actions still only reports that a unit test has failed without any indication of why:

PASS: argv_testdriver
FAIL: buffer_testdriver
PASS: crypto_testdriver

https://github.com/schwabe/openvpn/actions/runs/7385546211/job/20090514521

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 5
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-Attention: flichtenheld <[email protected]...>
Gerrit-Comment-Date: Tue, 02 Jan 2024 12:41:22 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

#5 lore raw
In reply to a message not in the archive

Attention is currently required from: plaisthos.

flichtenheld has posted comments on this change. ( http://gerrit.openvpn.net/c/openvpn/+/479?usp=email )

Change subject: forked-test-driver: Show test output always
......................................................................

Patch Set 5:

(1 comment)

Patchset:

PS5:

Quoted 1 line

The patch does not do what it promises to do. […]

Note that the forked-test-driver is only used for the system tests, not the UTs. Your concern was addressed by https://gerrit.openvpn.net/c/openvpn/+/485

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 5
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-Attention: plaisthos <[email protected]...>
Gerrit-Comment-Date: Tue, 09 Jan 2024 09:35:44 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: plaisthos <[email protected]...>
Gerrit-MessageType: comment

#6 lore raw
In reply to a message not in the archive

Attention is currently required from: flichtenheld.

plaisthos has posted comments on this change. ( http://gerrit.openvpn.net/c/openvpn/+/479?usp=email )

Change subject: forked-test-driver: Show test output always
......................................................................

Patch Set 7: Code-Review+2

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 7
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-Attention: flichtenheld <[email protected]...>
Gerrit-Comment-Date: Thu, 25 Jan 2024 10:32:47 +0000
Gerrit-HasComments: No
Gerrit-Has-Labels: Yes
Gerrit-MessageType: comment

#7 lore raw

PATCH v7 1/1 forked-test-driver: Show test output always

In reply to a message not in the archive

We want to see the progress, at least for slow tests like t_client.sh.

Change-Id I11e0091482d9acee89ca018374cb8d96d22f8514

---

This change was reviewed on Gerrit and approved by at least one developer. I request to merge it to master.

Gerrit URL: https://gerrit.openvpn.net/c/openvpn/+/479
This mail reflects revision 7 of this Change.

Acked-by according to Gerrit (reflected above):
Arne Schwabe <[email protected]...>

forked-test-driver +7 -2
index be73b80..6971dfb 100755
@@ -108,9 +108,14 @@
 # Test script is run here. We create the file first, then append to it,
 # to ameliorate tests themselves also writing to the log file. Our tests
 # don't, but others can (automake bug#35762).
+# OVPN changes:
+#  - add tee to see output of tests
+#  - needs portable pipefail mechanism
+estatusfile="${trs_file}.exit"
 : >"$log_file"
-"$@" >>"$log_file" 2>&1
-estatus=$?
+("$@" 2>&1; estatus=$?; echo $estatus >"$estatusfile") | tee -a "$log_file"
+estatus=$(cat "$estatusfile")
+rm -f "$estatusfile"
 
 if test $enable_hard_errors = no && test $estatus -eq 99; then
   tweaked_estatus=1
#8 lore raw

[Openvpn-devel] [PATCH applied] Re: forked-test-driver: Show test output always

Tested this in various build environments, including Ubuntu/MinGW, and GHA, and things still work & test "as before". Combined with the previous commit, we get useful (for me) "I can see what make check is doing all the time" behaviour back.

I'm still not convinced that this is a better way forward than just using the old test-driver and possibly tweaking Makefiles a bit, but "this does work for me", and automake upstream seems to think it's the way to go... thanks, Frank, for doing the actual work :-)

Your patch has been applied to the master branch.

commit e2ff9161e1b1b3e8c83bf01e3c488e0601834c0c

Author: Frank Lichtenheld
Date:   Thu Jan 25 12:01:22 2024 +0100
     forked-test-driver: Show test output always

     Signed-off-by: Frank Lichtenheld <[email protected]...>
     Acked-by: Arne Schwabe <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/[email protected]/msg28133.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering

#9 lore raw
In reply to a message not in the archive

Attention is currently required from: flichtenheld.

Hello plaisthos,

I'd like you to reexamine a change. Please visit

    http://gerrit.openvpn.net/c/openvpn/+/479?usp=email

to look at the new patch set (#10).

The change is no longer submittable: checks~ChecksSubmitRule is unsatisfied now.

Change subject: forked-test-driver: Show test output always
......................................................................

forked-test-driver: Show test output always

We want to see the progress, at least for slow tests like t_client.sh.

Change-Id I11e0091482d9acee89ca018374cb8d96d22f8514

---
M forked-test-driver

1 file changed, 7 insertions(+), 2 deletions(-)

git pull ssh://gerrit.openvpn.net:29418/openvpn refs/changes/79/479/10

forked-test-driver +7 -2
index be73b80..6971dfb 100755
@@ -108,9 +108,14 @@
 # Test script is run here. We create the file first, then append to it,
 # to ameliorate tests themselves also writing to the log file. Our tests
 # don't, but others can (automake bug#35762).
+# OVPN changes:
+#  - add tee to see output of tests
+#  - needs portable pipefail mechanism
+estatusfile="${trs_file}.exit"
 : >"$log_file"
-"$@" >>"$log_file" 2>&1
-estatus=$?
+("$@" 2>&1; estatus=$?; echo $estatus >"$estatusfile") | tee -a "$log_file"
+estatus=$(cat "$estatusfile")
+rm -f "$estatusfile"
 
 if test $enable_hard_errors = no && test $estatus -eq 99; then
   tweaked_estatus=1

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 10
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-Attention: flichtenheld <[email protected]...>
Gerrit-MessageType: newpatchset

#10 lore raw
In reply to a message not in the archive

cron2 has uploaded a new patch set (#12) to the change originally created by flichtenheld. ( http://gerrit.openvpn.net/c/openvpn/+/479?usp=email )

The following approvals got outdated and were removed:
Code-Review+2 by plaisthos

Change subject: forked-test-driver: Show test output always
......................................................................

forked-test-driver: Show test output always

We want to see the progress, at least for slow tests like t_client.sh.

Change-Id I11e0091482d9acee89ca018374cb8d96d22f8514

Message-Id: <[email protected]...>
URL: https://www.mail-archive.com/[email protected]/msg28133.html

---
M forked-test-driver

1 file changed, 7 insertions(+), 2 deletions(-)

git pull ssh://gerrit.openvpn.net:29418/openvpn refs/changes/79/479/12

forked-test-driver +7 -2
index be73b80..6971dfb 100755
@@ -108,9 +108,14 @@
 # Test script is run here. We create the file first, then append to it,
 # to ameliorate tests themselves also writing to the log file. Our tests
 # don't, but others can (automake bug#35762).
+# OVPN changes:
+#  - add tee to see output of tests
+#  - needs portable pipefail mechanism
+estatusfile="${trs_file}.exit"
 : >"$log_file"
-"$@" >>"$log_file" 2>&1
-estatus=$?
+("$@" 2>&1; estatus=$?; echo $estatus >"$estatusfile") | tee -a "$log_file"
+estatus=$(cat "$estatusfile")
+rm -f "$estatusfile"
 
 if test $enable_hard_errors = no && test $estatus -eq 99; then
   tweaked_estatus=1

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 12
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-MessageType: newpatchset

#11 lore raw
In reply to a message not in the archive

cron2 has submitted this change. ( http://gerrit.openvpn.net/c/openvpn/+/479?usp=email )

Change subject: forked-test-driver: Show test output always
......................................................................

forked-test-driver: Show test output always

We want to see the progress, at least for slow tests like t_client.sh.

Change-Id I11e0091482d9acee89ca018374cb8d96d22f8514

Message-Id: <[email protected]...>
URL: https://www.mail-archive.com/[email protected]/msg28133.html

---
M forked-test-driver

1 file changed, 7 insertions(+), 2 deletions(-)
forked-test-driver +7 -2
index be73b80..6971dfb 100755
@@ -108,9 +108,14 @@
 # Test script is run here. We create the file first, then append to it,
 # to ameliorate tests themselves also writing to the log file. Our tests
 # don't, but others can (automake bug#35762).
+# OVPN changes:
+#  - add tee to see output of tests
+#  - needs portable pipefail mechanism
+estatusfile="${trs_file}.exit"
 : >"$log_file"
-"$@" >>"$log_file" 2>&1
-estatus=$?
+("$@" 2>&1; estatus=$?; echo $estatus >"$estatusfile") | tee -a "$log_file"
+estatus=$(cat "$estatusfile")
+rm -f "$estatusfile"
 
 if test $enable_hard_errors = no && test $estatus -eq 99; then
   tweaked_estatus=1

--
To view, visit http://gerrit.openvpn.net/c/openvpn/+/479?usp=email
To unsubscribe, or for help writing mail filters, visit http://gerrit.openvpn.net/settings

Gerrit-Project: openvpn
Gerrit-Branch: master
Gerrit-Change-Id: I11e0091482d9acee89ca018374cb8d96d22f8514
Gerrit-Change-Number: 479
Gerrit-PatchSet: 12
Gerrit-Owner: flichtenheld <[email protected]...>
Gerrit-Reviewer: plaisthos <[email protected]...>
Gerrit-CC: openvpn-devel <[email protected]>
Gerrit-MessageType: merged